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Abstract 

We analyze the effects of noise on the permutation entropy of dy¬ 
namical systems. We take as numerical examples the logistic map and 
the Rossler system. Upon varying the noise strengthfaster, we find a 
transition from an almost-deterministic regime, where the permuta¬ 
tion entropy grows slower than linearly with the pattern dimension, to 
a noise-dominated regime, where the permutation entropy grows faster 
than linearly with the pattern dimension. We perform the same anal¬ 
ysis on experimental time-series by considering the stochastic spiking 
output of a semiconductor laser with optical feedback. Because of the 
experimental conditions, the dynamics is found to be always in the 
noise-dominated regime. Nevertheless, the analysis allows to detect 
regularities of the underlying dynamics. By comparing the results of 
these three different examples, we discuss the possibility of determin¬ 
ing from a time series whether the underlying dynamics is dominated 
by noise or not. 
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1 Introduction 


Ordinal analysis is a method of time series analysis that consists of com¬ 
puting the probabilities of ordinal patterns, which are dehned according to 
the ordering of D consecutive values in the series [1]. The entropy of these 
probabilities, referred to as permutation entropy, is a tool to detect possible 
regularities in the time series. In recent years, ordinal patterns and per¬ 
mutation entropy have been widely used to investigate complex dynamical 
systems [2,3]. They have been employed in the attempt to distinguish noise 
from chaos [4-7], to detect noise-induced order [8], serial correlations [9] 
and dependencies between two or more time series [10-16], among many 
other examples. Applications to experimental time series analysis include 
classihcation and discrimination of dynamical states in normal and epileptic 
EEG [17-20] and detection of heart rate variability under different physio¬ 
logical and pathological conditions [21-23]. 

Given this growing interest, it is relevant to understand the relation be¬ 
tween the permutation entropy and other complexity measures. In particular, 
a well-established way to characterize the production of information of a dy¬ 
namical system is the Kolmogorov-Sinai entropy h^s, see e.g. [24,25]. To 
compute hks, the time series is discretized by partitioning the phase space 
into regions and assigning a symbol to each region. Then one computes the 
probabilities of blocks, which are vectors of D consecutive symbols (more de¬ 
tails in the following sections). The entropy of the block probabilities is the 
block entropy. The Kolmogorov-Sinai entropy is hnally obtained as the rate 
of growth, for D ^ oo and in the limit of a very rehned partition, of the 
block entropy. 

Similarly to hks, one can introduce a permutation entropy rate as the rate 
of growth for D —)■ cx) of the permutation entropy. Both the permutation 
entropy and the Kolmogorov-Sinai entropy measure the “asymptotic” infor¬ 
mation rate of representations of the time series, the former with ordinal 
patterns (based in the relative order of consecutive values) and the latter 
with blocks (based on a partition of the phase space). The permutation 
entropy rate and hks are not only conceptually related: for piecewise mono¬ 
tone interval maps on the real line, they were shown to be equal [26]. This 
result has been later extended to a broad class of dynamics [27,28]. This 
equivalence is non-trivial considering, for example, that the number of total 
possible ordinal patterns grows with D as D\, while the number of blocks 
grows as where Q is the total number of symbols. The two quantities 
can be equal only thanks to the large number of forbidden ordinal patterns, 
strongly limiting the growth of the permutation entropy as D is increased. 

These mathematical results clarify that, under general hypotheses, per- 
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mutation and block entropies share the same asymptotic behavior. However, 
due to difficulties in reaching the asymptotic regime, this equivalence can be 
of little use in many practical cases. For example, it has been noted [26] 
that the rate of convergence of the permutation entropy to the Kolmogorov- 
Sinai entropy is extremely slow even for one-dimensional maps, while on the 
contrary, block entropies converge very quickly, see e.g. [24]. 

Comparing the two analyses becomes even more problematic for high¬ 
dimensional and/or noisy dynamics, such as typical experimental time-series. 
Consider for example the extreme case of a time series dominated by noise, in 
which all symbols are equally probable and temporal correlations are absent. 
In this case, the block entropy of length D is equal to Dln{Q), where Q is 
the total number of symbols, while the permutation entropy with patterns of 
length D is equal to ln(D!) ~ DlnD. This means that the block entropy is 
linear in D, with a slope ln((5), explicitly dependent on the chosen partition, 
which diverges only in the limit of a very rehned partition, Q —)■ oo. In 
contrast, the permutation entropy grows more than linearly, so that their 
asymptotic slope is inhnite. In both cases, the result is an inhnite entropy 
rate. However, to discover it, in the hrst case one needs to construct a very 
rehned partition. In the second case, one needs to reach large values of D to 
appreciate that the slope increases logarithmically. Both these tasks can be 
very difficult when analyzing a hnite time series due to statistical limitations. 

Our goal is to get a better understanding of how noise inhuences the 
permutation entropy. To this aim, we analyze simulated and experimental 
time series. We mostly focus on permutation entropy as the effect of noise 
on block entropies is fairly well understood, see e.g. [29,30]. We hrst analyze 
time series generated from the logistic map and from Poincare sections of 
the three-dimensional Rossler system. We conclude with an experimental 
example of output intensity data recorded from a semiconductor laser with 
optical feedback. 

2 Methods 

2.1 Numerical data 

We consider two dynamical systems: the one-dimensional logistic map and 
the three-dimensional Rossler system. In both cases, we study the ehect of 
adding to the dynamical equations a Gaussian white noise, with {^t) = 
0 and temporal correlation We considered also the case of 

observational noise (not shown), where the dynamics is deterministic but the 
noise ahects the observation, obtaining very similar results. 
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2.1.1 Logistic map 


xt+i = 4xt(l - xt) + a^t, (1) 

where Xt is the state of the system at iteration t and a is the noise strength. 
In order to constrain the variable Xt in the interval [0,1], the values of 
that would lead to Xt+i > 1 or Xt+i < 0 are simply discarded and redrawn. 
Thus, the noise is temporally uncorrelated, but not purely Gaussian due 
to this truncation effect. To investigate the variation of the permutation 
entropy with the noise strength, we computed the permutation entropy, for 
each value of a, from time series of length iV = 12 x 10^. We have also studied 
other nonlinear one-dimensional maps (Tent, Bernoulli and Quadratic) and 
obtained very similar results to those of the logistic map (results not shown). 

2.1.2 Rossler system 

The Rossler equations read 

X = -Y - Z + a^{t), 

Y = X + aY (2) 

Z = b + Z{X-c) 

where {X, Y, Z} are the states of the system at time t, a is the noise strength 
and {a,b,c} are the local parameters set at {0.1, 0.1, 18.0}, respectively. 

In order to apply the symbolic methods (ordinal patterns or blocks) we 
need to discretize the dynamics. Instead of employing temporal sampling 
[32], we introduce a Poincare section [33] at X = 0, and analyze the time 
intervals between consecutive crossings of the Poincare plane. For each value 
of a, the permutation entropy is computed from time-series of X = 12 x 10^ 
data points. 

2.2 Experimental data 

Experimental data was recorded from the output intensity of a semiconduc¬ 
tor laser with optical feedback operating in the low-frequency fluctuations 
(LFFs) regime. In this regime, the laser intensity displays sudden and ap¬ 
parently random dropouts, followed by gradual recoveries. This spiking dy¬ 
namics has received considerable attention because the intensity dropouts 
are induced by stochastic effects and deterministic nonlinearities. The opti¬ 
cal feedback introduces a delay which renders the system in principle inhnite 
dimensional. Therefore, the laser in the LFF regime generates complex fluc¬ 
tuations that, because of the stochastic and high-dimensional nature of the 
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underlying dynamics, are suitable to be investigated by means of complexity 
measures such as the permutation and block entropies. 

The experimental setup is the same as in [31] and uses a 650 nm AlGalnP 
semiconductor laser (SONY SLD1137VS) with optical feedback. The feed¬ 
back was given through a mirror placed 70 cm apart from the laser cavity, 
with a round trip of 4.7 ns. The feedback was controlled using a neutral 
density hlter that can adjusts the light intensity injected into the laser. The 
laser has a solitary threshold current of Ith = 28.4 niA. The temperature 
and current of the laser were stabilized using a combi controller Thorlabs 
ITC501 with an accuracy of 0.01 C and 0.01 niA, respectively. The current 
used during the experiment was / = 29.3 mA and the temperature was set 
at r = 17 C. The neutral density hlter was adjusted so that the threshold 
reduction due to feedback was about 7%. The signal was captured using 
a photo detector (Thorlabs DET210) connected to a FEMTO HSA-Y-2-40 
ampliher and registered with a 1 GHz digital oscilloscope (Agilent Inhniium 
DSO9104A) with 0.2 ns of sampling. The intensity time series were acquired 
from the oscilloscope by a Lab VIEW program that uses a threshold to de¬ 
tect the times when the intensity drops, and calculates the time intervals 
between successive threshold crossings (in the following, referred to as inter¬ 
dropout-intervals, IDIs). We recorded in this way time series of more than 
10® consecutive IDIs. 

2.3 Methods of analysis 

We compare two different methods to transform a time-series, x{t) = {a;(l), 
x{2)... x(Y)}, into a sequence of symbols, s{t): ordinal patterns and blocks. 

In both cases, one needs to choose a dimension D for dehning vectors 
made up of consecutive entries of the time series, i.e. {x{i),x{i + l)^ ..., x{i + 
D — 1)}. Ordinal patterns and blocks differ by the way in which the entries of 
these vectors are transformed into symbols. Ordinal patterns classify them 
according to the ranking (from the largest to the smallest value) of the D 
entries in the vectors. The total number of ordinal patterns of length D is 
then equal to the number of permutations, D\. For example, with D = 2 
there are two ordinal patterns: x{ti) > x(tj+i) corresponding to the ordinal 
pattern ‘01’ and x(ti) < x(ti+i) corresponding to the ordinal pattern ‘10’. 

For blocks, the phase space is hrst divided into Q regions, associating a 
symbol to each region. Blocks represent all vectors in which each value of 
the time series correspond to the same symbol. For example, let us consider 
the time series x(t) = {0.1, 0.6,0.7, 0.3}, and partition the phase space into 
the two regions [0,0.5) and [0.5,1], associating to them the symbols 0 and 
1 respectively. With D = 2, the blocks associated to the time series are 
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Figure 1: Permutation entropy {H) as a function of the size of the ordinal 
pattern (D) and the noise strength (a) for data generated from the Logistic 
map. (a) H vs D for a = 1 x 10“^^ (stars), a = 2 x 10“^ (triangles), 
a = 5 X 10“^ (inverted triangles), a = 0.1 (circles), a =1 (pentagons) and 
Hmax = In D\ (solid line), (b) H versus aior D = 2 (stars), D = 3 (triangles), 
D = A (inverted triangles), D = 5 (circles), D = 6 (pentagons), D = 7 
(squares) and D = 8 (diamonds). 

{ 01 , 11 , 10 }. 

Then, the permutation entropy [1] or the block entropy [24,25] are simply 
the entropy of the frequency pi of the different patterns in the time series 

M 

Hd =-J2pi\npi, ( 3 ) 

i 

where M is the number of possible patterns: for permutation entropy, M = 
D!, while for block entropy, M = Q^. 

3 Results 

Figure la displays the permutation entropy, H, vs the dimension of the 
ordinal patterns, D, computed from time series of the logistic map at different 
noise strengths. It can be observed that H increases monotonically with D, 
regardless of the noise strength. As the noise increases, H approaches its 
maximum value, corresponding to equally probable ordinal patterns, Hmax = 
InD! (solid black line). Note that at a = 1 (pentagons) the values of H 
is already very close to Hmax- Figure lb displays iL as a function of the 
noise strength a. A clear transition from low-noise to high-noise can be 
observed, for a value of the noise strength approximately independent of D. 
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The difference between the values of the entropies at low and high noise 
becomes more pronounced as D increases. 

To further investigate this transition, Fig. (2a) displays the difference 
Hd — Hd-i as a function of D, for various values of noise strength. As 
before, we indicate with a thin black line the noise-dominated limit in which 
all patterns are equiprobable, H^, — H£,_i = (InD! — ln(D — 1)!). In the 
opposite limit of almost-deterministic, as D grows the expected value of 
Hd — Hd-i is the Kolmogorov-Sinai entropy [26], which for a one-dimensional 
chaotic map is equal to the Lyapunov exponent A. In the case of logistic map 
for a local parameter set at 4 one has A = In 2, indicated by the thick black 
line. As shown in detail in Fig. (2c), we identify three possibilities: 

• a almost-deterministic regime in which Hd — Hd^i decreases for large 

D, 

• a noise-dominated regime in which Hd — Hd^i increases for large iA, 

• an intermediate regime in which Hu — remains nearly constant 

with D. 

In principle, this qualitative feature of the permutation entropy can be 
applied to experimental time series to assess whether the dynamics is domi¬ 
nated by noise or by the deterministic dynamics. 

We remind that this distinction can not be done for the block entropy, 
as in this case H^, — H£,_i is necessarily a decreasing function of D (see 
e.g. [34,35]). This fundamental difference between permutation entropy and 
block entropy can be appreciated by comparing the left and right panels of 
Fig. (2). 

For the analysis of the Rossler data, we considered the Poincare map X = 
0, shown in Fig. (3a), and analyzed the sequence of time-intervals between 
consecutive crossings. Figure (3b) shows the difference Hd — Hd-i vs D, for 
different values of a. The solid line indicates the expected value if all ordinal 
patterns were equally probable. Hr, — Hd-i = InD! — ln(iA — 1)!. Because 
of the high level of stochasticity, we calculate the conhdence interval that is 
consistent with the null hypothesis of equally probable ordinal patterns: in 
Fig. (3b) the gray region represents the expected value ±3cr, where a is the 
standard deviation calculated for a hundred surrogated (shuffle) time-series. 

Before testing the method in experimental data we want to investigate 
how the choice of the Poincare section influences the results. We consider a 
Poincare section in the plane Z = /5, as shown in Fig. (4a), and varying f3 in 
the range [0.05 — 26.7], for a hxed value of a = 0. In this case, to discretize 
the time series, we analyze the time values when the trajectory intersects the 
Poincare section and Z grows. 
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Figure 2: Comparison of the entropy computed from ordinal patterns, and 
the entropy computed from the blocks, for the Logistic map. The difference 
iLfl — is plotted vs. the dimension of the ordinal patterns (a,c) and of 

the blocks (b,d) for various values of noise strength [the noise strengths are 
as in Fig. (la)]. In panels (a) and (b) the solid lines indicate the asymptotic 
values for low noise (thick) and high noise (thin). Panel (c) and (d) display 
a detail of (a) and (b). 
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Figure 3: (a) Rossler attractor and Poincare section in X = 0. (b) Permuta¬ 
tion entropy difference, Hij — vs the dimension of the ordinal patterns, 

D, for noise strength a = [0 (star), 0.8 (triangle), 1.6 (inverted triangle), 2.4 
(circle), 3.2 (pentagon), 4 (square)]. The gray region indicates the values of 
Hd — i^D-i that are consistent with equally probable ordinal patterns (see 
text for details). For the smallest value of alpha, Hd — shows a non¬ 

monotonic behavior, while for higher values of the noise strength, Hu — H£,_i 
grows monotonically with D. 


Figure (4b) displays the difference vs. D, for different values of 

/?. We can see that the difference Hij — Hij_i increases with jS. This is due to 
the fact that, as f3 is increased, consecutive values in the time-series become 
increasingly uncorrelated, similarly to when increasing the noise strength. 
On the contrary, for the minimum value of [5, the variation oi Hd — Ho-i 
with D is resemblant to the behavior under almost-deterministic observed in 
Fig. (3b). 

Finally, we analyze experimental data from the laser output intensity, 
displayed in Fig. (5a). To discretize the data we consider the thresholds 
indicated with horizontal lines in Fig. (5a), and analyzed the time intervals 
between consecutive threshold-crossings [7,9,31]. Figure (5b) displays the 
difference Hu — Hu-i vs. D, for different thresholds. Note that Hu — Hu-i 
varies with the threshold in a similar way as in Fig. (4b): as the threshold 
decreases, correlations between consecutive dropouts are lost. 

For all the thresholds, Hu — Hu-i grows monotonically with D. The rea¬ 
son is that the empirical time series is very noisy and the “almost-deterministic” 
regime is not seen, not even for the highest threshold. Nevertheless, the val¬ 
ues of Hu — Hu-i lie outside the gray region that indicates values consistent 
with equally probable ordinal patterns. This reveals that the sequence of 
intensity dropouts are not completely uncorrelated, and thus, this method 
can determine regularities also in very noisy data. 
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Figure 4: (a) Rossler attractor and Poincare section placed m. z = [i. (b) 
Permutation entropy difference, ~ ^d-i, vs the dimension of the ordinal 
patterns, D, for j3 = 0.05 (stars), (3 = 6.7 (triangles), [3 = 13.4 (inverted 
triangles), (3 = 20.0(circles), {3 = 26.7 (pentagons). The behavior is qualita¬ 
tively similar to the one observed in Fig. (3b). 




Figure 5: (a) Experimentally recorded time-series for the output intensity 
of a semiconductor laser, which operates in the low-frequency fluctuations 
(LFFs) regime, induced by self time-delayed optical feedback. The hori¬ 
zontal lines indicate the thresholds used to detect the dropout times, (b) 
Permutation entropy difference, — Hd_i, vs the dimension of the ordinal 
patterns, for different thresholds: —0.5 (stars), —2 (inverted triangles) and 
—4 (pentagons). 
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4 Conclusion 


We have studied the influence of noise in the permutation entropy of dynam¬ 
ical systems, considering both, simulated data and experimental data. In 
the simulated data, when increasing the noise strength, a transition between 
a almost-deterministic regime and a noise-dominated regime was clearly ob¬ 
served. The noise value at which this transition occurs is roughly independent 
of the size D of the ordinal pattern. 

In the almost-deterministic regime, the permutation entropy grows al¬ 
most linearly or sub-linearly with D. This behavior is qualitatively similar 
to that of the block entropy. However, to observe a quantitative equivalence 
it is often needed to analyze extremely long time series, which can be com¬ 
putationally unfeasible even for relatively simple dynamical systems. In the 
noise-dominated regime, the growth is faster than linear, i.e. the differences 
Hd — Hd-i increase with D. In principle, this fact can be used to deter¬ 
mine whether the dynamics is in a noise-dominated or a almost-deterministic 
regime from an experimental time series where the noise strength can not 
be externally tuned. However, care must be taken in interpreting the re¬ 
sults, as extracting a one-dimensional time series from a purely deterministic 
high-dimensional time series via a Poincare map can lead to ordinal pat¬ 
terns which look effectively noisy, as we demonstrated in the example of the 
Rossler systems. This fact reflects the well-known difficulties of distinguish¬ 
ing deterministic dynamics from noise when dealing with high-dimensional 
systems [29]. 
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